The Divergence of Reinforcement Learning Algorithms with Value-Iteration and Function Approximation

This paper gives specific divergence examples of value-iteration for several major Reinforcement Learning and Adaptive Dynamic Programming algorithms, when using a function approximator for the value function. These divergence examples differ from previous divergence examples in the literature, in that they are applicable for a greedy policy, i.e. in a "value iteration" scenario. Perhaps surprisingly, with a greedy policy, it is also possible to get divergence for the algorithms TD(1) and Sarsa(1). In addition to these divergences, we also achieve divergence for the Adaptive Dynamic Programming algorithms HDP, DHP and GDHP.Comment: 8 pages, 4 figures. In Proceedings of the IEEE International Joint Conference on Neural Networks, June 2012, Brisbane (IEEE IJCNN 2012), pp. 3070--307

Numerical analysis of rapid drawdown: applications in real cases

In this study, rapid drawdown scenarios were analyzed by means of numerical examples as well as modeling of real cases with in situ measurements. The aim of the study was to evaluate different approaches available for calculating pore water pressure distributions during and after a drawdown. To do that, a single slope subjected to a drawdown was first analyzed under different calculation alternatives, and numerical results were discussed. Simple methods, such as undrained analysis and pure flow analysis, implicitly assuming a rigid soil skeleton, lead to significant errors in pore water pressure distributions when compared with coupled flow-deformation analysis. A similar analysis was performed for the upstream slope of the Glen Shira Dam, Scotland, and numerical results were compared with field measurements during a controlled drawdown. Field records indicate that classical undrained calculations are conservative but unrealistic. Then, a recent case of a major landslide triggered by a rapid drawdown in a reservoir was interpreted. A key aspect of the case was the correct characterization of permeability of a representative soil profile. This was achieved by combining laboratory test results and a back analysis of pore water pressure time records during a period of reservoir water level fluctuations. The results highlight the difficulty of predicting whether the pore water pressure is overestimated or underestimated when using simplified approaches, and it is concluded that predicting the pore water pressure distribution in a slope after a rapid drawdown requires a coupled flow-deformation analysis in saturated and unsaturated porous media.Peer ReviewedPostprint (published version

Criteria for rapid sliding I.: a review of Vaiont case

Vaiont slide has been represented by a model of two interacting evolutive wedges. Mass conservation during the motion implies that the upper wedge transfers mass to the lower one through an internal shearing plane. The model respects available in situ observations. It was formulated in dynamics terms. Outcomes of the analysis are the determination of safety factors of the valley before dam impoundment, and the calculation of run-out distance once the motion starts. Rock strength degradation as motion develops has also been included. This degradation, even if it is intense, was unable to explain the very high estimated landslide velocities.Peer ReviewedPostprint (author's final draft

Slope stability in slightly fissured claystones and marls

The final publication is available at Springer via http://dx.doi.org/10.1007/s10346-014-0526-5The paper deals with the behavior of some characteristic soft rocks found in the Iberian Peninsula. In geological terms, they belong to Tertiary basins, the Keuper period and the Jurassic-Cretacic transition. The discussion is organized around the following aspects: (a) the intact material and its brittle behavior; (b) the weathering action of atmospheric events; (c) the persistent discontinuities and scale effects; and (d) the modification of strength after failure. In all cases, instability phenomena are addressed in connection with several case histories. Regarding material brittleness and the initial stress state, two cases of first time failures are discussed. Practical implications concerning the selection of operative strength will be given. Field observations of the relevance of weathering and, also, on the rate of weathering, are given for a Weald claystone. Field observations emphasize the importance of sharp transitions between weathered and intact (or slightly weathered) levels. A recent long-term laboratory investigation on the nature of degradation will be summarized. Macroscopic variables such as stiffness and tensile strength have been found to be uniquely predicted by a degradation law in terms of the accumulated plastic deformations. Persistent discontinuities and, in particular, sedimentation planes play a dominant role to explain slope failures not related to the shallow failures, usually associated with weathered profiles. The strength of discontinuities in a Weald formation was investigated by means of tests performed at two scales. Finally, the evolution in time of residual strength induced by chemical actions, associated with groundwater flow, is highlighted in connection with actual field data of unstable slopes.Peer ReviewedPostprint (author's final draft

Criteria for rapid sliding II.: thermo-hydro-mechanical and scale effects in Vaiont case

Thermally induced excess pore pressures have been included into a two-wedge evolutive model of Vaiont landslide. The problem requires the solution of a system of four coupled balance equations for the shear bands and the surrounding rock as well as the joint equation of motion of the entire slide. The model predicts the high velocities observed and is consistent with other data (slide geometry, residual strength, and conditions on the sliding surface). The interpretation of a sensitivity analysis suggests that there exists a threshold permeability band, in the range 10- 8 to 10- 10 m/s, which separates potentially fast motions from slow motions. This conclusion is maintained if the scale of the landslide is reduced.Peer ReviewedPostprint (author's final draft

El control judicial de los secretos de Estado en España. A propósito de las exportaciones de armamento

En este trabajo se plantea una propuesta de sistematización de los mecanismos de control de los secretos de Estado en el ordenamiento jurídico español. Tres son los parámetros jurídicos de control: 1) que la información clasificadaafecte a la seguridad y defensa del Estado, 2) un juicio de probabilidad sobre el posible daño o riesgo a la seguridad y defensa, y 3) la ponderación de la defensa nacional con otros derechos o intereses en conflicto. Estos parámetros de control se aplican a un caso concreto: la información relativa a lasexportaciones de armamento que son autorizadas por la Administración. El resultado del control es que debería considerarse ilegal la clasificación de losdatos relativos al comercio de armas